User interface (UI) is often one of the most important parts of a computer program because it determines how easily a user can communicate with the program. A powerful program with a poorly designed UI has little value. Text-based and graphical user interfaces (GUIs) that use windows, icons, and pop-up menus have become standard on personal computers. Text-based UIs as well as GUIs typically use an input device, such as a keyboard, mouse or stylus, to provide user input and control the movement of a cursor or pointer on a display screen.
Touch-sensitive surfaces are rapidly becoming more common in computing devices. A natural input device for computing devices with touch-sensitive surfaces is user's finger. They are very convenient as they allow a user to make natural gestures familiar to the user in other contexts, such as by entering handwriting using a stylus. Many of these devices also allow input to be provided by a user's fingertip. The term touch-sensitive surface or device will be used herein to refer to such surfaces or devices that are configured to detect the touch of any type of “stylus” such as a stylus, stylus type device or a user's finger, fingers, hand or hands.
As portable electronic devices become more compact, and the number of functions performed by a given device increase, it has become a significant challenge to design a user interface that allows users to easily interact with various devices including multifunction devices. This challenge is particularly significant for handheld portable devices, which have much smaller screens than desktop or laptop computers. This situation is unfortunate because the user interface is the gateway through which users receive not only content but also responds to user actions or behaviors, including user attempts to access a device's features, tools, and functions. Some portable communication devices (e.g., PDAs, mobile telephones, sometimes called mobile phones, cell phones, cellular telephones, smart phones, and the like) have resorted to adding more pushbuttons, increasing the density of push buttons, overloading the functions of pushbuttons, or using complex menu systems to allow a user to access, store and manipulate data. These conventional user interfaces often result in complicated key sequences and menu hierarchies that must be memorized and accessed by the user.
Many conventional user interfaces, such as those that include physical pushbuttons, are also inflexible. This may prevent a user interface from being configured and/or adapted by either an application running on the portable device or by users. When coupled with the time consuming requirement to memorize multiple key sequences and menu hierarchies, and the difficulty in activating a desired pushbutton, such inflexibility is frustrating to most users.
To avoid problems associated with pushbuttons and complex menu systems, portable electronic devices may use touch screen displays that detect user gestures on the touch screen and translate detected gestures into commands to be performed. However, user gestures may be imprecise; a particular gesture may only roughly correspond to a desired command. Other devices with touch screen displays, such as desktop computers with touch screen displays, also may have difficulties translating imprecise gestures into desired commands.
Accordingly, there is a need for touch screen display electronic devices with more transparent and intuitive user interfaces for translating imprecise user gestures into precise, intended commands that are easy to use, configure, and/or adapt. Such interfaces increase the effectiveness, efficiency and user satisfaction with portable multifunction devices. The need to elaborate methods of touch screen device user's gesture recognition and flexible touch commands has been recognized in both industry and academia. Numerous inventions have been reported in that area. For example, in the U.S. Pat. No. 7,519,223 “Recognizing gestures and using gestures for interacting with software applications” by Dehlin et al, an interactive display table is described that senses and infers natural hand or finger positions, or movement of an object, to detect gestures. Specific gestures are used to execute applications, carry out functions in an application, create a virtual object, or do other interactions, each of which is associated with a different gesture.
Other solutions, including a U.S. Pat. No. 7,479,949 “Touch screen device, method, and graphical user interface for determining commands by applying heuristics” by Jobs et al, assigned to Apple Inc., are also attempting to address this problem. U.S. Pat. No. 7,479,949 covers many touch screen commands related to the use of iPhone. Inventions of that patent are based on detecting one or more finger contacts with the touch screen display, applying one or more heuristics to the one or more finger contacts to determine a command for the touch screen device, and processing the command.
As more software applications are being implemented on computing system with touch screen interface, it is clear that any set of similar applications may require specific interpretation of touch commands and gestures recognized by touch sensitive devices. In other words, the same gestures or touch commands may mean different things for different applications. Moreover, new touch commands and gestures will be implemented as touch screen devices gain more popularity. While many commands exist already (as is seen from the above mentioned examples), there is a need to have better and more intuitive commands for specific applications. These commands will help to eliminate push buttons and “drop down” interfaces that use up space on a touch screen. Clearly such commands provide significant benefit to devices with small touch screen. In this invention we describe a system that utilizes a set of new touch commands. We also describe a method by which these commands can be correctly detected and interpreted. As will be seen later, these commands are naturally linked to certain data processing operations illustrated in the preferred embodiment.
We illustrate these commands' usage in data processing applications where each data unit has structured content identified by an appropriate tag (e.g., as in XML documents). Specific example of a data used for illustration purposes is a description of a real estate property for sale. We will call such description a data record. Clearly there are many such data records, and often a potential buyer (user) wants to analyze two or more of such records. Invented commands implemented on a system with a touch screen interface greatly improve the process of analyzing such data.
What is also needed is a system that can understand simple and intuitive user gestures and can interpret them as appropriate touch screen commands that allow the user to align, sort, view commonality or difference of multiple data records. The system would also be displaying on a touch screen resulting operation on selected data records according to a tag associated with data records. The tag could be related to time, location, distance, or any specific attribute (e.g., number of bedrooms for the data records describing homes for sale), etc. Applications taking advantage of such commands will be disclosed in embodiments below. Such applications include but are not limited to analysis of any structured data records such as real estate records, quarterly sales report of a retail store, description of new/used cars for sale, etc. Any person skilled in the art will recognize that the touch screen commands presented in this invention can be used in many other applications.
Therefore, present invention disclosure describes a computing device with a touch screen display that is able to recognize new touch screen commands improving usability of the above mentioned applications.